In the development of the autonomous vehicle, sensing devices are essential components for future autonomous vehicles. Various sensing devices could be integrated for providing more reliable sensing data and more precise information of a surrounding within any type of environment. Although various object detection techniques may reliably and accurately detect designated objects, inconsistent sensor performances over diversified sensing modalities within different environments may decimate the performance of most of existing detection techniques which are simultaneously used but sub-optimally integrated when a process of performing synchronized data acquisition.
Because of current limitations in hardware restricted detection performances, detection techniques would experience various difficulties. For example, less than reliable detection results may occur for an object in a high speed velocity due to low data sampling rate. Using uniform specifications on the detection results may result in including pixels that are not part of the objects to be detected as a detected object(s) due to operation in the lowest coordinate system. Fine path planning may miss detected object(s) due to operations in the lowest data resolution.
FIG. 1A illustrates various sensors which could be used to generate a depth map. Such sensors may include not limited to a LiDAR, a Stereo Camera, and a time-of-flight camera (ToF Camera). In further detail, Table 1 shows a comparison of characteristics among the sensors that can produce depth information of surroundings of an autonomous vehicle. Values shown in Table 1 are only shown for exemplary purposes as specific values may vary based on design considerations.
TABLE 1ConstraintsLiDARStereo CameraToF Camerasensor typeactivepassiveactive + passive effective range100 meters             f      cam        ×          l      base        disparity  10 meters accuracyhighlowmoderateresolutionsparsedensemid-densefield of view360 degrees≤360 degrees≤360 degreessampling ratemoderateslowmoderate
According to Table 1 and the prior elaboration, the state of the art sensors for depth sensing could be achieved by a LiDAR sensor which typically has a 360 degrees field of view, the farthest detection range, and the highest depth accuracy in comparison to the other instruments such as a stereo camera or a ToF camera. The data resolution and the sampling rate of a LiDAR, however, could be limited based on the several factors.
For example, the number of beams used to acquire depth volume could be limited. FIG. 1B illustrates a comparison between low resolution LiDAR data and high resolution LiDAR data. The low resolution LiDAR data is, for example, a projected point cloud volume taken by LiDARs with a fewer number of laser beams (e.g. 16-beams), and the high resolution LiDAR data is, for example, a projected point cloud volume taken by LiDARs with a greater number of laser beams (e.g. 64-beams). Among the current version of various LiDAR transducers, trade-off relationships could be apparent; for example, more affordable LiDARs would have fewer number of beams (i.e. 16-beams LiDAR illustrated in FIG. 1B), and a fewer number of beams in LiDARs would generate a fewer number of points/seconds, yet a fewer number of beams LiDAR would consume less power. With these trade-offs, there could be a few possible implications including, for example, difficulties to recognize object with sparse point clouds because of fewer point clouds acquired as the corresponding object speeds up (as illustrated in FIG. 1B).
Among the current version of various LiDAR transducers, trade-off relationships could be apparent, for example, nearer objects may have more point clouds, and smaller-sized objects may have fewer point clouds. With these trade-offs, there are few possible implications including, for example, difficulties to recognize smaller-sized objects since the number of point clouds would be extremely low for meaningful analysis, and there could be even fewer point clouds to be acquired as the corresponding object speeds up.
Based on the aforementioned trade-off relationships, a depth up-sampling method which increases detection reliability by a transformation from sparse point clouds to dense point clouds could be essential. There could be a few goals that can be achieved from a depth up-sampling technique. For example, more precise path planning could be achieved since each pixel could take on a depth value as detection results could be more finely segmented rather than just using a bounding box. Also, better detection accuracy could be achieved by using a dense depth map-the detection algorithm which may more reliably detect objects despite the object's relative position, size, or velocity (i.e. acceleration). The overall system could be affordable system since the depth up-sampling may enable the utilization of a dense depth map acquired from low-cost LiDARs rather than utilizing a depth map acquired from high-end LiDARs.